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Abstract 

Current education standards in the U.S. re¬ 
quire school students to read and under¬ 
stand complex texts from different subject 
areas (e.g., social studies). However, such 
texts usually contain figurative language, 
complex phrases and sentences, as well as 
unfamiliar discourse relations. This may 
present an obstacle to students whose na¬ 
tive language is not English — a growing 
sub-population in the US. 1 One way to 
help such students is to create classroom ac¬ 
tivities centered around linguistic elements 
found in subject area texts (DelliCarpini, 
2008). We present a web-based tool that 
uses NLP algorithms to automatically gen¬ 
erate customizable linguistic activities that 
are grounded in language learning research. 

1 Introduction 

Recent educational standards adopted by several 
states in the U.S. (CCSSO, 2010) explicitly em¬ 
phasize the need for students to read progressively 
more complex texts in different subject areas, to 
prepare for college and careers. However, to accom¬ 
plish this, learners need to have a grasp of linguistic 
features related to vocabulary, word senses, figura¬ 
tive language, English conventions, and discourse 
structures. 

English language learners (ELLs) generally 
struggle to acquire English language skills: reading, 
writing, speaking, and listening. These learners 
could be disadvantaged further if there were simply 
an increase in the complexity of texts without con¬ 
current scaffolding to help them with the demands 
likely to enter the curriculum as a result of the 
new Standards (Coleman and Goldenberg, 2012). 

^ee http://www.neela.us/files/uploads/ 
9/growingLEP_0809.pdf 


This suggests the need for subject area teachers to 
incorporate a more linguistically-based approach 
to support content comprehension (Christie, 1989; 
Christie, 1999). Yet, teachers often lack the train¬ 
ing necessary to identify English language features 
that may challenge diverse groups of ELLs (Slavin 
and Cheung, 2004; Walqui and Heritage, 2012). 

In this paper, we present Language Muse , an 
open-access, web-based tool that can address these 
needs. 2 Specifically, Language Muse can help 
subject area teachers support ELLs by automat¬ 
ically generating customizable activities derived 
from actual texts used in their classrooms. The 
activities are generated using several existing NLP 
algorithms and are designed to help ELLs with mul¬ 
tiple aspects of language learning needed to support 
content comprehension: vocabulary, grammatical 
structures, and discourse & text organization. 

Although Language Muse is related to existing 
work in the NLP literature on automatic question 
generation (Mitkov and Ha, 2003; Brown et al., 
2005; Heilman and Smith, 2010), it can generate 
multiple activities for teachers’ own texts, cover a 
significantly larger set of language constructs, and 
offer teachers much more customizability. 

In subsequent sections, we first provide a de¬ 
scription of the Language Muse NLP. Next, we 
describe how teachers interact with the backend 
and create activities. Finally, we present the results 
of a survey conducted with actual ELL teachers, 
and conclude with future work. 

2 NLP Backend 

Language Muse relies on a backend that uses NLP 
techniques and resources to identify a variety of 
linguistic features contained in an input text. The 
features being identified can be categorized as: (a) 
lexical entities (single word and multi-word expres¬ 
sions), (b) syntactic structure, and (c) rhetorical 

2 http://languagemuse.10clouds.com. 
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(a) Lexical Entities 

Cognates 

Identified using a manually-created dictionary that was verified by a native 
Spanish speaker (most U.S. ELLs speak Spanish as their native language). 

Academic Words & 

Definitions 

Words that describe complex and abstract concepts, and are used across disci¬ 
plines, e.g., analyze , benefit. Identified using a manually-created list and defi¬ 
nitions extracted using the Wordnik API (http : / /developer . wordnik . 
com). 

Frequent Concepts 

Words that appear repeatedly across the text. Identified using a heuristic that 
measures repetitions across paragraphs. 

Multiword Expres¬ 
sions 

Idioms, Phrasal Verbs, etc. Identified using a rank-ratio based collocation 
detection algorithm trained on the Google Web IT n-gram corpus (Futagi et al., 
2008) 

Contractions 

Identified using regular expressions defined on constituency parses. 

Complex Words 

Morphologically complex or irregular verbs. Identified using a rule-based 
morphological analyzer (Leacock and Chodorow, 2003). 

Morphological 

Variants 

Generated using an algorithm that first over-generates variants using rules and 
then filters using co-occurrence statistics computed over Gigaword. 

Synonyms 

Generated using a thresholded combination of WordNet (Fellbaum, 1998), a 
distributional thesaurus (Lin, 1998), and SMT-based paraphrases (Bannard and 
Callison-Burch, 2005). 

Antonyms 

Generated using WordNet. 

Homonyms 

Generated using a manually-created list (Burstein et al., 2004). 

(b) Syntactic Structure 

Note: All modules below use regular expressions on constituency parses. 

Relative Clauses 

Sentences containing an explicit relative clause. 

1+ Clauses 

Sentences containing 1 independent clause and >= 1 one dependent clause. 
Note that this can also include sentences with relative clauses. 

Complex NPs 

Noun phrases with a hyphenated adjective or a prepositional phrase modifier. 

Complex Verbs 

Verb phrases with >= 2 verb forms, e.g., will have gone , plans to leave 

(c) Rhetorical and Discourse Relations 

Note: All modules below use an adapted rule-based discourse analyzer (Burstein et al., 1998). 

Cause-Effect 

Terms indicating a cause-effect relation between text segments, e.g., “The 

discovery of fossils of tropical plants in Antarctica led to the hypothesis that 
>> 

Compare-Contrast 

Terms indicating a comparison or contrast between text segments, e.g., “He was 
a wise and patient leader; however , his son ...” 

Evidence & Details 

Terms indicating specific evidence or details between text segments, e.g., “Re¬ 
cent theories, such as the influence of plate tectonics on the movement of 
continents , have ...” 


Table 1: The inventory of features provided by the backend NLP engine in Language Muse. 
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and discourse relations. Before we describe each 
category in detail, it is important to note that since 
the primary use case for Language Muse is to help 
teachers plan appropriate classroom activities for 
ELLs, it is important for the automatically gener¬ 
ated activities to be as accurate as possible. There¬ 
fore, for many of the features, we rely on manually 
crafted resources, either directly or indirectly as a 
filter for a noisier statistical approach. 

Table 1 shows the linguistic features that our 
system can identify in the three aforementioned 
categories and provides a brief description of the 
backend module is used to generate it. 

3 Activity Generation 

In this section, we describe how users interact with 
Language Muse , i.e., how they can automatically 
generate linguistic activities for any text and cus¬ 
tomize them to their own liking. Language Muse 
is completely free to use for all teachers. Teachers 
request an account using the form on the web site 
and receive their login information via email. 

Once a teacher logs into Language Muse , she 
can get started either by choosing one of the 33 
texts that we provide across three different content 
areas (English Language Arts, Science, and So¬ 
cial Studies), or by uploading her own classroom 
text (in plain text/.doc/.docx formats). The system 
currently limits the texts to 5000 words. All texts 
uploaded by a teacher are saved into her personal 
library for later re-use. 

The text is then sent to the NLP backend for pro¬ 
cessing, which returns a JSON object containing 
all identified (or generated) linguistic features. At 
that point, the teacher can generate any of the avail¬ 
able linguistic activities, each of which is based on 
one of the linguistic features. All activities were 
designed based on input from ELL content-area 
teachers. There are a total of 24 activities, grouped 
according to whether an activity is word-based, 
sentence-based, or paragraph-based. This form 
of hierarchical grouping is based on ELL litera¬ 
ture which suggests that each level in the hierarchy 
presents distinct challenges and opportunities for 
language learning. Table 2 shows a few of the 
available activities and provides a brief description. 

3.1 Recommended Activities 

Based on the number of feature instances detected 
by the backend, Language Muse may recommend 
certain activities over others to the teacher. For 
example, if there were more words with synonyms 


but only a few cause-effect terms, it might recom¬ 
mend the Synonyms in Paragraphs activity but not 
the Cause/Effect Relationships activity. Some ac¬ 
tivities may also be unavailable since no instances 
of the corresponding linguistic feature could be 
detected by the backend. Language Muse makes 
a visual distinction between recommended activi¬ 
ties, possible activities, and unavailable activities 
as shown in Figure 1. Clicking on an activity shows 
its description and a sample question. 

3.2 Same Feature, Multiple Activities 

Some activities are based on the same underlying 
linguistic feature but use it differently, depending 
on their level. For example, a word-level activity 
asks students to match words in one list to words 
in another list based on how similar they are in 
meaning. That activity uses automatically gener¬ 
ated synonyms for the words in the text and then 
automatically populates the two lists - one with 
the original words and the other with the generated 
synonyms. There is a similar paragraph-level cloze 
activity that shows students a paragraph from the 
text and asks them to replace pre-identified words 
with their synonyms such that the meaning is un¬ 
changed. This activity uses the same underlying 
feature — automatically generated synonyms — 
but presents it differently. This exposes ELLs to a 
different part of the language construct. 

3.3 Automatically-generated Answers 

The questions for all activities are automatically 
generated based on linguistic features in the text. 
However, for 15 of those activities, Language Muse 
also automatically populates an answer key for 
the teacher. For example, for the word-based syn¬ 
onym activity described in §3.2, we know which 
pairs of words in the two lists match each other 
since the synonyms were automatically generated. 
Automatically-generated answers reduce the time 
that a teacher needs to edit an activity for her class¬ 
room. See Table 2 for additional examples. 

3.4 Customizability 

It is impossible for Language Muse to always pro¬ 
vide exactly what every teacher is looking for. 
Therefore, almost all aspects of the activities it gen¬ 
erates can be customized to suit a teacher’s needs. 
Among other things, the teacher can choose to: 

• edit the instructions shown to the students, 

• hide any or all automatically chosen 
words/sentences/paragraphs, 
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Sentence activities 

Multiple Clause Sentences. Shows multi-clause sentences and asks students to break them 
up into two or more shorter sentences. Although the sentences for the activity are identified 
automatically in the text, the answers are not generated automatically. Example: Organelles are 
structures visible within a cell that have their own structure. (1) Organelles are structures 
visible within a cell. (2) Organelles have their own structure. 

Cause/Effect Relationships. Shows sentences containing causal relationships and asks students 
to identify the cause, the effect, and the connector word that denotes the causal relationship. 
The sentences with causal relationships in the text are identified automatically but only the 
connector word part of the answer is automatically generated. Example: Off the coast of Canada, 
commercial cod fishing had to stop because the population of cod collapsed. => The population 
of cod collapsed off the coast of Canada (cause), Commercial cod fishing had to stop (effect), 
because (connector). 

Homonyms in Sentences. Shows sentences with blanks and asks students to fill in the right 
word from a list that contains homonyms as distractors. Examples can be seen in Figure 2. 

Paragraph activities 

Variant Word Forms in Paragraphs. Uses inflectional and derivational word variants gener¬ 
ated by the backend. Shows students a paragraph of text with blanks and asks them to fill in the 
right morphological variant. Answers and distractors are automatically generated. Example: 

Scientists suspect that there are more than 10 million _ (different/difference) types of life 

forms on Earth. 

Phrasal Verbs. Asks students to pick the correct preposition to complete the phrasal verbs 
found in the paragraph. Answers and distractors are automatically generated. Example: People 
usually think _ (at/on/of) the heart, lungs and brain as vital organs. 


Table 2: A subset of the linguistic activities available to teachers in Language Muse. 


Q Search for an activity... 

0 Paragraph activities 

★ 

Variant word forms in paragraphs 


Summary practice with paragraphs 


Summary practice with paragraphs 
with key term hints 

—k 

Synonyms in paragraphs 


★ 

Recommended activity: Contains 5 or more questions related to language skill 

V" 

Activity contains 1-4 questions related to language skill 


Unable to generate activity related to language skill 


Review sample activities from the list, then click Create Activity to 
generate activities based on your text 


Phrasal Verbs 

— ★ 

Cognates (Spanish) 


Figure 1: Activities where 5 or more questions can be generated are recommended by Language Muse 
and marked with a star. Activities with fewer than 5 questions are not marked but can still be chosen by 
the teacher. Activities with no available questions are greyed out and cannot be chosen. 
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Figure 2: The Homonyms in Sentences activity generated from a Language Arts text on Virginia Woolf. 
The questions, answers, and distractors are all automatically generated. 


Activity 

<^>Hide all paragraphs 

Paragraph 1 



Virginia Woolf said," 
fiction." 

a woman must have money and a room of her own if she is to .. 


List of answers 

® write <g> Hide 

O wrote <g> Hide 

O writing <g£> Hide 

O written <$£> Hide 

O writes <g> Hide 



Add new answer 


It was 1928 and Virginia Woolf spoke these words from two papers she had . 


List of answers 

O writes <$£> Hide 
(•) written <g> Hide 

O write <$g> Hide 

O writing <g> Hide 

O wrote <g> Hide 



Add new answer 



Figure 3: An example of a customizable multiple choice activity (Variant Word Forms in Paragraphs). 
Teachers can (a) change the generated correct answer, (b) hide any generated distractors, (c) add their own 
answers, and (d) hide a paragraph entirely or use it only as context without generating questions. 
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• edit the automatically generated answers or 
add her own, and 

• edit the list of automatically generated detrac¬ 
tors for multiple choice questions. 

Figure 3 illustrates this by showing the Variant 
Word Forms in Paragraphs activity in edit mode. 

4 Teacher Survey 

We wanted to evaluate whether activities generated 
by Language Muse are useful to teachers. To do 
this, we worked with seventeen 6th-8th grade teach¬ 
ers who taught English language arts, science, and 
social studies. Four teachers had been teaching for 
two years or less, five for 3-9 years, and the rest for 
>10 years. All but two currently had responsibil¬ 
ity for teaching ELLs and eight had been teaching 
ELLs for > 5 years. We asked them to examine 
9 different activities from Language Muse and tell 
us whether they would consider using them in their 
classrooms. Figure 4 shows the results of our sur¬ 
vey which are encouraging for a first version. 
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Figure 4: A heatmap showing the results of our 
teacher survey. Each cell shows the number of 
middle school teachers responding to whether they 
would use the corresponding activity in classrooms. 

5 Conclusions & Future Work 

We presented Language Muse , an open-access, 
web-based tool that can help content-area teachers 
support ELL students with content comprehension. 
We are currently working on the next version which 
will allow students to log into Language Muse to 
complete any activities assigned to them by their 
teachers and also receive feedback. All develop¬ 
ment on Language Muse continues to be informed 
by frequent and detailed interactions with teachers. 
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